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Abstract 

The group T of automorphisms of the polynomial 

n(x, y, z) = x 2 + y 2 + z 2 - xyz - 2 

is isomorphic to 

PGL(2,Z) x (Z/2©Z/2). 

For t G R, the F-action on K _1 (t)nlR 3 displays rich and varied dynamics. The 
action of T preserves a Poisson structure defining a T -invariant area form on each 
fill For t < 2, the action of T is properly discontinuous on the four con- 
tractible components of DM 3 and ergodic on the compact component (which 

is empty if t < —2). The contractible components correspond to Tcichmiiller spaces 
of (possibly singular) hyperbolic structures on a torus M . For t = 2, the level set 
K _1 (t) n R 3 consists of characters of reducible representations and comprises two er- 
godic components corresponding to actions of GL(2,Z) on (R/Z) 2 and R 2 respectively. 
For 2 < t < 18, the action of T on « _1 (t) fl R 3 is ergodic. Corresponding to the Fricke 
space of a three-holed sphere is a T -invariant open subset ft C R 3 whose components 
are permuted freely by a subgroup of index 6 in T . The level set k _1 (£) fl R 3 intersects 
il if and only if t > 18, in which case the T action on the complement (K _1 (()nl 3 )-H 
is ergodic. 
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Introduction 

Let M be a compact oriented surface of genus one with one boundary com- 
ponent, a one-holed torus. Its fundamental group it is free of rank two. Its 
mapping class group 7r (Homeo(M)) is isomorphic to the outer automorphism 
group Out(7r) of tt and acts on the space of equivalence classes of representa- 
tions 7r — ► SL(2) . We investigate the dynamics of this action on the set of real 
points on this moduli space. 

Corresponding to the boundary of M is an element K £ it which is the com- 
mutator of free generators X, Y G it. By a theorem of Fricke [8, 9], the moduli 
space of equivalence classes of SL(2, C) -representations naturally identifies with 
affine 3-space C 3 , via the quotient map 

Hom(vr,SL(2,C)) — ► C 3 









■ tr(ppO) " 


p 1 — ► 


y 




tr(p(Y)) 




z 




ti(p(XY))_ 



In terms of these coordinates, the trace tr p(K) equals: 

k(x, y, z) := x 2 + y 2 + z 2 — xyz — 2 

which is preserved under the action of Out(7r) . The action of Out(7r) on C 3 is 
commensurable with the action of the group V of polynomial automorphisms 
of C 3 which preserve k (Horowitz [17]). (Compare also Magnus [22].) 

Theorem Let k(x, y, z) = x 2 + y 2 + z 2 — xyz — 2 and let t G M.. 

• For t < — 2, the group T acts properly on K _1 (i) fl M 3 ; 

• For —2<t<2, there is a compact connected component Ct of K~ l (t) n 
M 3 upon which T acts ergodically; T acts properly on the complement 
(K _1 (i)ni 3 ) -C t ; 

• For t = 2, the action of T is ergodic on the compact subset k _1 (2) n 
[— 2,2] 3 and the action is ergodic on the complement k~ 1 {2) — [— 2,2] 3 ; 

• For 18 > t > 2, the group T acts ergodically on ^(t) n M 3 ; 

• For t > 18, the group T acts properly and freely on an open subset 
Clt C K _1 (t) n R 3 , permuting its components. The T -action on the com- 
plement of Qt is ergodic. 
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The proof uses the interplay between representations of the fundamental group 
and hyperbolic structures on M. The dynamics breaks up into two strikingly 
different types: representations corresponding to hyperbolic structures com- 
prise contractible connected components of the the level sets whereas 
representations which map a simple nonperipheral essential loop to an ellip- 
tic element, comprise open subsets of upon which T is ergodic. Thus 
nontrivial dynamics accompanies nontrivial topology of the moduli spaces. 

In his doctoral thesis [31], G. Stantchev considers characters corresponding to 
representations into PGL(2,R) (that is, actions preserving H 2 but not pre- 
serving orientation on H 2 ). For t < — 14 (respectively t > 6) characters of 
discrete embeddings representing hyperbolic structures on 2-holed projective 
planes (respectively 1-holed Klein bottles) give wandering domains in the cor- 
responding level set. For — 14 < t < 2 the T-action on the corresponding level 
set is ergodic, and for t < — 14, the action is ergodic on the complement of the 
wandering domains corresponding to Fricke spaces of 2-holed projective planes. 




(a) Level set k = —2.1 (b) Level set k = 1.9 

Figure 1 

Notation and terminology We work in Poincare's model of the hyperbolic 
plane H 2 as the upper half-plane. We denote the identity map (identity matrix) 
by I. Commutators are denoted: 

[A,B] = ABA~ l B~ l 

and inner automorphisms are denoted: 

i a : x i — ► gxg' 1 . 
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For any group it we denote the group of all automorphisms by Aut(7r) and the 
normal subgroup of inner automorphisms by Inn(7r) . We denote the quotient 
group Aut(7r)/Inn(7r) by Out(7r). A compact surface S with n boundary 
components will be called n -holed. Thus a one-holed torus is the complement 
of an open disc inside a torus and a three-holed sphere (sometimes called a 
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"pair-of-pants" ) is the complement of three disjoint discs inside a sphere. 
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1 The modular group and the moduli space 

In this section we define the modular group and the moduli space for the one- 
holed torus. The modular group is isomorphic to 

7r (Homeo(M)) Out(vr) GL(2,Z) 

and the moduli space is affine space C 3 . We explain how the invariant function 
k originates. We describe various elements of the modular group and how they 
act on the moduli space. 
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1.1 The mapping class group 

Let M denote a compact connected orientable surface of genus one with one 
boundary component. Since attaching a disc to M yields a torus, we refer 
to M as a one-holed torus. The mapping class group of M is the group 
7To(Homeo(M)) of isotopy classes of homeomorphisms of M . We investigate 
the action of 7ro(Homeo(M)) on the moduli space of flat SL(2) -connections on 
M. 

1.1.1 Relation to m(M) 

Choose a basepoint xq £ M and let ir := ni(M;xo) . Any homeomorphism of 
M is isotopic to one which fixes xq and hence defines an automorphism of n. 
Two such isotopic homeomorphisms determine automorphisms of it differing 
by an inner automorphism, producing a well-defined homomorphism 

N: 7r (Homeo(M)) — ► Out(?r) = Aut(7r)/Inn(vr). (1.1.1) 

If M is a closed surface, then Dehn (unpublished) and Nielsen [30] proved 
that N is an isomorphism. (See Stillwell [32] for a proof of the Dehn-Nielsen 
theorem.) When dM ^ 0, then each component diM determines a conjugacy 
class Ci of elements of tti(M). The image of N consists of elements of Out(7r) 
represented by automorphisms which preserve each Ci . 

Let M be a one-holed torus. Its fundamental group admits a geometric redun- 
dant presentation 

n = (X,Y,K\[X,Y]=K) (1.1.2) 

where K corresponds to the generator of ni(dM). Of course, n is freely gen- 
erated by X,Y. 

1.1.2 Nielsen's theorem 

The following remarkable property of M is due to Jakob Nielsen [29] and does 
not generalize to other hyperbolic surfaces with boundary. For a proof see 
Magnus-Karrass-Solitar [23], Theorem 3.9 or Lyndon-Schupp [21], Proposition 
5.1. 

Proposition 1.1.1 Any automorphism of the rank two group 

n=(X,Y,K | K = [X,Y]) 
takes K to a conjugate of either K itself or its inverse K^ 1 . 
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An equivalent geometric formulation is: 

Proposition 1.1.2 Every homotopy-equivalence M — > M is homotopic to a 
homeomorphism of M . 

Thus the homomorphism (1.1.1) defines an isomorphism 

N: vr (Homeo(M)) — ► Out(vr). 

1.2 The structure of the modular group 

We say that an automorphism of tt which takes K to either K or K~ x is 
normalized. The normalized automorphisms form a subgroup Aut(*7r, K) of 
Aut(vr). 

Let 4> be an automorphism of tt. Proposition 1.1.1 implies that 4>(K) is conju- 
gate to K ±x . Thus there exists an inner automorphism i g such that 

<f>(K)=gK*g-\ 

that is 

i~ 1 O (j) £ Aut(ir,K) 

where e = ±1. (In fact, e = det ( h ([</>])) , where h is the homomorphism defined 
by (1.2.1) below.) Since the centralizer of K in tt equals the cyclic group (K) , 
the automorphism <p determines the coset of g modulo (K) uniquely. 

We obtain a short exact sequence 

1 — ► (ix) ^ Aut(vr, K) — > Out(vr) — > 1 

The action on the homology H±(M; Z) = Z 2 defines a homomorphism 

h: Out(vr) — > GL(2,Z). (1.2.1) 

By Nielsen [29], h is an isomorphism. (Surjectivity follows by realizing an 
element of GL(2,Z) as a linear homeomorphism of the torus 1R 2 /Z 2 . See 
Lyndon-Schupp [21], Proposition 4.5 or Magnus-Karrass-Solitar [23], Section 
3.5, Corollary N4.) We obtain an isomorphism 

H: GL(2,Z) — > Aut{ir,K)/{i K ). 

Restriction of the composition fx o h to Aut(7r, K) equals the quotient homo- 
morphism 

Aut(vr,K) — > Aut(vr, K)/{i K ). 
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2 Structure of the character variety 



2.1 Trace functions 



The relevant moduli space is the character variety, the categorical quotient of 
Hom(7r, G) by the G-action by inner automorphisms, where G = SL(2,C). 
Since n is freely generated by two elements X and Y, the set Hom(7r, G) of 
homomorphisms tt — ► G identifies with the set of pairs (£, rj) G G xG, via the 
mapping 

Hom(i, G) — > G x G 

p^(p(X),p{Y)). 

This mapping is equivariant respecting the action of G on Hom(-7r, G) by 

g : pi — > Lgop 

and the action of G on G x G by 

g: (£,rf)\ — > {gig' 1 , grig' 1 ). 

The moduli space Hom(7r, G)//G consists of equivalence classes of elements 
of Hom(7r, G) = G x G where the equivalence class of a homomorphism p is 
defined as the closure of the G-orbit Gp. Then Hom(-7r, G)//G is the categorical 
quotient in the sense that its coordinate ring identifies with the ring of G— 
invariant regular functions on Hom(7r,G). For a single element g € G , the 
conjugacy class Inn(G)(g) is determined by the trace t = ti(g) if t ^ ±2. That 
is, 

Inn(G)( 5 ) =tr- 1 (t). (2.1.1) 

For t = ±2, then 



tr- x (±2) = {±1} \J Inn(G) 





"1 


1" 









1 



(2.1.2) 



By Fricke [8] and Fricke-Klein [9] , the traces of the generators X, Y, XY param- 
etrize Hom(7r, G)//G as the affine space C 3 . As X and Y freely generate tt, 
we may identify: 

Hom(f, G) < — ► G x G 

p ^( p (X),p(Y)) 

The character mapping 

X- Hom(vr, G)//G — C 3 





"x(p)" 




■ tr(p(X)) " 




y(p) 




tr(p(y)) 




.«(p). 




tv(p(XY)) 
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is an isomorphism. (Compare the discussion in Goldman [12], 4.1, [13], Sections 
4-5 and [14].) 

For example, given (x,y,z) £ C 3 , the representation p defined by 



P{X) 



x — 1 
1 



o r 1 

-C y 



(2.1.3) 



satisfies \{p) = { x iVi z ) where ( € C is chosen so that 

Conversely, if (x,y,z) £ C 3 then x~ l { x -,V-, z ) consists of a single G-orbit if 
and only if n(x,y,z) / 2 where k is defined below in (2.1.4). (This is also 
the condition that p is an irreducible representation of it . Compare Lubotzky- 
Magid [20], Brumfiel-Hilden [4], Culler-Shalen [6].) 

For any word w(X, Y) , the function 

Hom(7r, G) — > C 

[p]^tr(p(w(X,Y)) 

is G-invariant. Hence there exists a polynomial f w (x,y,z) € C[x,y, z] such 
that 

tv(p(w(X,Y)) = f w (x(p),y(p),z(p)). 

A particularly important example occurs for w(X,Y) = [X, Y] = K, in which 
case we denote f w (x,y,z) by n(x,y,z). By an elementary calculation (see, for 
example [14]), 

tr(p(K)) = k(x, y, z) := x 2 + y 2 + z 2 - xyz - 2. (2.1.4) 

The level set k -1 (£) consists of equivalence classes of representations p: it — > 
G where p(K) is constrained to lie in tr _1 (t). (Compare (2.1.1) and (2.1.2) 
above.) 

2.2 Automorphisms 

Let G = SL(2, C) . The group Aut(-7r) acts on the character variety 
Hom(vr,G)//G by: 

and since 



P°h = L p(y) ° P 
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the subgroup Inn(7r) acts trivially. Thus 0ut(7r) acts on 

Hom(vr,G)//G ^ C 3 
and since an automorphism <f> of tt is determined by 

(<P(X),^(Y)) = ( Wl (X,Y),w 2 (X,Y)), 
the action of <fi on C 3 is given by the three polynomials f Wl , f W2 , /, 



X 


fw\ { x 


y, 




y !- 


~ * ftll2 { X 


y, 


z) 


z 


_fw3 { X 


y, 


*). 



where ws(X,Y) = wi(X,Y)w2(X,Y) is the product word. Hence Out(7r) acts 
on C 3 by polynomial automorphisms. Nielsen's theorem (Proposition 1.1.1) 
implies that any such automorphism preserves k : C 3 — > C , that is 

K {fwi { x i y-, z )t fw2 { x i y-> z )i fw\w2 { x iV-i z )) = k(x, y, z). 



2.2.1 Sign-change automorphisms 

Some automorphisms of the character variety are not induced by automor- 
phisms of tt. Namely, the homomorphisms of tt into the center {±1} C G form a 
group acting on Hom(7r, G) by pointwise multiplication. Let ( G Hom(7r, {±1}) 
and p G Hom(7r, G) . Then 

C-P- 7^C(7)/>(7) (2-2.1) 
is a homomorphism. This defines an action 

Hom(vr, {±1}) x Hom(7r, G) — ► Hom(7r, G). 
Furthermore, since {±1} is central in G and K G tt is a commutator 

(C-p)(K) = p(K). (2.2.2) 

Since tt is free of rank two, 

Hom(vr, {±1}) ^ Z/2 x Z/2. 

The three nontrivial elements (0, 1), (1, 0), (1, 1) of Z/2 x Z/2 act on represen- 
tations by o"i,o"2, 03 respectively: 



(Ji • p: 



a 2 - p: 



X ^p(X) 

X — 

Y — p(y) 
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0-3 ■ p: 



The corresponding action on characters is: 



-p(X) 
-P(Y) 





X 




X 


(0-1)* : 


y 




-y 




z 




—z 




X 




— X 


(0-2)* ■ 


y 




y 




z 




—z 




X 




—x 


(0-3)* : 


y 




-y 




z 




z 



We call this group the group of sign-changes and denote it by S. 
Evidently S preserves n(x,y,z). (Alternatively apply (2.2.2).) 



2.2.2 Permutations 

Since k(x, y, z) is symmetric in x, y, z the full symmetric group S3 also acts 
on C 3 preserving k. Unlike S, elements of S3 are induced by automorphisms 
of 7r . The group of all linear automorphisms of (C 3 , k) is generated by S and 
S3 and forms a semidirect product £ x S3 . 

S3 is actually a quotient of T. The projective line F l (Z/2) over Z/2 has three 
elements, and every permutation of this set is realized by a projective transfor- 
mation. Since Z/2 has only one nonzero element, the projective automorphism 
group of P x (Z/2) equals GL(2,Z/2) and GL(2,Z/2) ^ S 3 . The action of 
GL(2,Z) on (Z/2) 2 defines a homomorphism 

GL(2,Z) — ► GL(2,Z/2) = S 3 

whose kernel GL(2,Z)^ is generated by the involutions 



(2.2.3) 



"1 " 




"1 -2" 




"1 " 


-1 


5 


-1 




2 -1 



and —I G GL(2,Z)/ 2 ). The sequence 

PGL(2,Z) (2) :=GL(2,Z) (2) /{±I} 
is exact. The kernel T( 2 ) of the composition 

T — ► PGL(2,Z) - 



PGL(2,Z) — > S 3 



S, 
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equals the semidirect product 

PGL(2,Z) (2) x (Z/2©Z/2) 

and T is an extension 

r (2) — > r — > 6 3 . 

(See Goldman-Neumann [15] for more extensive discussion of T and its action 
on the the set of complex points of the character variety.) 

2.2.3 Other automorphisms 

For later use, here are several specific elements of T . See also the appendix to 
this paper. 

The elliptic involution is the automorphism 

X 1 — ► x- 1 

Y 1 — ► Y~ l 

corresponding to —I £ GL(2, Z) and acts trivially on characters. 

The Dehn twist about X is the automorphism tx G Aut(7r) 

X 1 — > X 
Y\ — >YX 

inducing the automorphism of characters 



X 




X 


y 




xy — z 


z 




V 



and corresponds to 

I I] €GL(2,Z). 
The Dehn twist about Y is the automorphism ry <G Aut(7r) 

x 1 — >iy 
y 1 — ► y 

inducing the automorphism of characters 



X 




xy — z 


y 




y 


z 




x 



Geometry & Topology, Volume 7 (2003) 



Action of the modular group 



455 



and corresponds to 



1 
1 1 



€ GL(2,Z). 



The quadratic reflection Q z is the automorphism 

X i — ► X 
Y i — ► Y- 1 

inducing the automorphism of characters 



X 




X 


y 




V 


z 




xy — z 



and corresponds to 



1 

-1 



€ GL(2,Z). 



2.3 Reducible characters 

Any representation p having character in k~ 1 (2) is reducible, as can be checked 
by direct calculation of commutators in SL(2, C) . Namely, let £, r] G SL(2, C) . 
We may write 

a b 



V 



c d 



where ad — be = 1 . By applying an inner automorphism, we may assume that 
£ is in Jordan canonical form. If £ is diagonal, 



A 
A- 1 



then 



tr[£,r?] = 2 + 6c(A-A- 1 ) 2 



implies that if tr[£,r/] = 2, then either £ = ±1 or 6c = (so £ is upper- 
triangular or lower-triangular). Otherwise 



1 s 
1 



(where s / 0), in which case 



tr[£,7/] = 2 + s 2 c 2 > 2 



(2.3.1) 
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so tr[£,77] = 2 implies that c = and rj is upper-triangular. Thus if tv(p(K)) = 
2, then p is conjugate to an upper-triangular representation. 

We may replace p by its semisimplification, that is the upper-triangular matri- 
ces by the corresponding diagonal matrices (their semisimple parts) to obtain 
a representation by diagonal matrices having the same character: 

o r 1 



p(X) = 
p(Y) = 
p(XY) = 



(2.3.2) 



rj * 

7]- 1 

c * ' 

o r 1 



where £77 = (. Thus 



(2.3.3) 



77 + 77" 



-1 



1/ = 

z = C + C 

(2.3.3) corresponds to the following factorization (2.3.4) of 
k(x, y, z) — 2 = x 2 + y 2 + z 2 — xyz — 4. 



Under the embedding 



[x, y, z\ 



defined by (2.3.3), the polynomial n(x,y,z) — 2 factors: 

«(£+r\»7+»r\c + c 1 )-2 = 
r 2 (i - ^o(i - r V)(i - e^ou - ^r 1 ) 



(2.3.4) 



Given (x,y,z) with n(x,y,z) = 2, the triple (£,77, £) is only defined up to an 
action of Z/2 © Z/2. Namely the automorphisms 



X- 1 
Y 



act on (f , 77, C) by: 





-r 1 






< 7/ 1- 


— » 77 


< 


77 




-r 1 







y- 1 



r 1 



x- 1 
y- 1 



c 



respectively. 
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2.4 The Poisson structure 



As in Goldman [13], Section 5.3, the automorphisms preserving k are unimod- 
ular and therefore preserve the exterior bivector field as well 

~ 2ir 2 dx dy dz 

which restricts to (the dual of) an area form on each level set which is 

invariant under Out(7r). We shall always consider this invariant measure on 
K _1 (t)nlR 3 . In the case when has a rational parametrization by an 

affine plane, that is, when t = 2, the above bivector field is the image of the 
constant bivector field under the parametrization: 



C x C 



d_ d_ 



6 S,+V _)_ £-(£ + »?) 



2it 2 Z 



The bivector field E defines a Poisson structure on the moduli space C 3 for 
which k defines a Casimir function. 

(The coefficient 2ir 2 occurs to normalize the area of the compact component 
Ck of reducible SU(2) -characters to 1.) Here we study the action of Out(7r) 
on the set K _1 (t) PI M 3 of R -points of K _1 (t) where t G R, with respect to this 
invariant measure. 



2.5 The orthogonal representation 

To describe the structure of the character variety more completely, we use a 3- 
dimensional orthogonal representation associated to a character (x,y,z) £ C 3 . 
(Compare Section 4.2 of Goldman [12] and Brumfiel-Hilden [4].) This will be 
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used to identify real characters as characters of representations into the real 
forms SL(2,R) and SU(2) of SL(2,C). 

Consider the complex vector space C 3 with the standard basis {ei,e2,e 3 } and 
bilinear form defined by the symmetric matrix 

'2 z y 
z 2 x 
y x 2 

Since 

det(B) = -2(n(x,y,z) - 2) 

(where n(x,y,z) is defined in (2.1.4)), the symmetric bilinear form is nonde- 
generate if and only if k(x, y, z) ^2. 

Assume n(x,y,z) ^ 2 so that B is nondegenerate. Let SO(C 3 ,B) = SO(3,C) 
denote the group of unimodular linear transformations of C 3 orthogonal with 
respect to B. The local isomorphism 

SL(2,C) — ► SO(C 3 ,B) 

is a surjective double covering, equivalent to the adjoint representation (or the 
representation on the second symmetric power of C 2 ), and is unique up to 
composition with automorphisms of SL(2, C) and SO (3, C) . 

For any vector v <G C 3 such that M(v,v) / 0, the reflection 

R v : x^x-2^4v (2.5.1) 
B(v,v) 1 ; 

is a B-orthogonal involution. 

Let Z/2*Z/2*Z/2 be the free product of cyclic groups (Ri) , where i = 1,2,3, 
and Rf = 1. The homomorphism 

7T — > Z/2*Z/2*Z/2 
X ^ R 2 R 3 
Y ^ R 3 R! 

embeds 7r as an index two subgroup of Z/2*Z/2*Z/2. 

Lemma 2.5.1 Let p G Hom(-7r, G) satisfy n(x,y,z) ^ 2. The restriction to ir 
of the representation p: Z/2*Z/2*Z/2 — > SO(C 3 ,B) defined by R { \ — ► R Ci 
is conjugate to the composition $ o p: tt — > SO(C 3 ,B) . 

See Section 4.2 of Goldman [12], or Brumfiel-Hilden [4] for a proof. 
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2.6 The set of M-points 

A real character is the character of a representation into a real form of G. 
(Proposition III. 1.1 of Morgan-Shalen [26]). The above orthogonal representa- 
tion gives an explicit form of this result. 

Suppose (x,y,z) € M 3 and K,(x,y,z) ^ 2. Then the restriction of B to R 3 
is a nondegenerate symmetric M -valued bilinear form (also denoted B), which 
is either indefinite or (positive) definite. Let SO(R 3 ,B) denote the group of 
unimodular linear transformations of M 3 orthogonal with respect to B. 

There are two conjugacy classes of real forms of G, compact and noncompact. 
Every compact real form of G is conjugate to SU(2) and every noncompact 
real form of G is conjugate to SL(2,R). Specifically, if B is positive defi- 
nite, then SO(R 3 ,B) is conjugate to SO(3) and $ _1 (SO(R 3 ,B)) is conjugate 
to SU(2). If B is indefinite, then SO(R 3 ,B) is conjugate to either SO(2, 1) 
or SO(l,2) (depending on whether n(x,y,z) > 2 or n(x,y,z) < 2 respec- 
tively) and <E> _1 (SO(R 3 ,B)) is conjugate to SL(2,R). Thus every real character 
(x, y, z) <G R 3 is the character of a representation into either SU(2) or SL(2, R) . 

When x,y,z G R, the restriction of B to R 3 is definite if and only if —2 < 

x,y,z < 2 and n(x,y,z) < 2. 

The coordinates of characters of representations tt\(M) — ► SU(2) satisfy: 

-2<x,y,z<2, x 2 + y 2 + z 2 - xyz - 2 < 2 

and, for fixed t € [—2,2], comprise a component of K _1 (t)nR 3 . The other 
four connected components of ft _1 (t)(lR 3 consist of characters of SL(2,R)- 
representations. These four components are freely permuted by S. For t < — 2, 
all four components of the relative character variety K _1 (t)nR 3 consist of 
characters of SL(2, R) -representations and are freely permuted by E . For t > 2 , 
the relative character variety K~ 1 (t) n R 3 is connected and consists of characters 
of SL(2, R) -representations. 

The two critical values ±2 of tr: G — ► C deserve special attention. When 
t = — 2, a representation p G Hom(7r,G) with tv p(K) = — 2 is a regular point 
of the mapping 

E K : Hom(f, G) — > G 

p ^p(K). 

Such a representation is a regular point of the composition 

tr oE K : Hom(vr, G) — > C 
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unless p(K) = —I. In the latter case (when \p(X),p(Y)] = —I), the represen- 
tation is conjugate to the quaternion representation in SU(2) 



P{X) 



i 
-% 



P(Y) 



-1 

1 



(2.6.1) 



In both cases, G acts locally freely on the subset (tr oEk) 1 (— 2) of Hom(7r, G) 
with quotient k _1 ( — 2). 

Now consider the case t = 2. As in Section 2.3, the R -points of k _1 (2) corre- 
spond to reducible representations, and in fact are characters of representations 
with values in the Cartan subgroups (maximal tori) of the real forms SU(2) and 
SL(2, R) . Every Cartan subgroup of SU(2) is conjugate to U(l) , and every Car- 
tan subgroup of SL(2, R) is conjugate to either SO (2) or SO (1, 1) . Characters 
of reducible SU(2) -representations form the compact set 

C K = K ' 1 (2)n[-2,2f 

which identifies with the quotient of the 2-torus U(l) x U(l) by {±1} under 
the extension (2.3.3). 

Characters of reducible SL(2, R) -representations comprise four components (re- 
lated by Ci = aiCo for i = 1, 2, 3): 

C = k _1 (2) n {[2, oo) x [2, oo) x [2, oo)^ , 
Ci = k _1 (2) n ([2, oo) x (-oo, -2] x (-oo, -2]^ , 
C 2 = k _1 (2) n (Voc, -2] x [2, oo) x (-oo, -2]^ , 
C 3 = k-\2) n {{-oo, -2] x (-oo, -2] x [2, oo)^) , 

each of which identifies with the quotient of R + x R + by {±1} under (2.3.3). 
These four components are freely permuted by E . There is a compact compo- 
nent of equivalence classes of SL(2, R) -representations which are irreducible but 
not absolutely irreducible — that is, although R 2 is an irreducible n -module, 
its complexification is reducible. In that case the representation is conjugate 
to a representation in SO (2) . This component agrees with the component Ck 
consisting of characters of reducible SU(2) -representations. 
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3 Hyperbolic structures on tori (t < 2) 

The topology and dynamics change dramatically as t changes from t < — 2 to 
t > 2. The level sets t < —2 correspond to Fricke spaces of one-holed tori 
with geodesic boundary. The level set for t = — 2 consists of four copies of the 
Teichmiiller space of the punctured torus, together with {(0, 0, 0)} . (The origin 
is the character of the quaternion representation in SU(2) .) The origin is fixed 
under the action, while T acts properly on its complement in 2). (How- 

ever the action on the set of complex points of 2) is extremely nontrivial 

and mysterious; see Bowditch [3].) 

The level sets for —2 < t < 2 correspond to Teichmiiller spaces of singular 
hyperbolic structures with one singularity, as well as a (compact) component 
consisting of characters of unitary representations. Except for the component 
of unitary representations, there are four components, freely permuted by S. 
Except for the component of unitary representations, the T-action is proper. 

3.1 Complete hyperbolic structures (t < —2) 

The theory of deformations of geometric structures implies that T acts properly 
on certain components of (t) . When t < 2 , the moduli space k~ 1 (t) contains 

4 contractible noncompact components, freely permuted by S. (When —2 < 
t < 2, an additional compact component corresponds to SU(2) -representations.) 
These contractible components correspond to SL(2, M) -representations, and 
each one identifies with the Teichmiiller space of M, with certain boundary 
conditions. Specifically, if t < —2, then these components correspond to hyper- 
bolic structures on int(M) with geodesic boundary of length 2 cosh _1 (— 1/2) . 
For t = —2, these components correspond to complete hyperbolic structures on 
int(M) and identify with the usual Teichmiiller space %m ■ For —2<t<2, 
these components correspond to singular hyperbolic structures on a torus whose 
singularity is an isolated point with cone angle 

6 = 2cos" 1 (-t/2). 

3.2 Complete structures and proper actions 

We begin with the case t < —2. Then each component of K _1 (t) fl M 3 paramet- 
rizes complete hyperbolic structures on int(M) with a closed geodesic parallel 
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to dM having length 2 cosh 1 (— 1/2). Thus the union 

U K-\t) 
t<-2 

consists of four copies of the Fricke space of M. This space contains equiva- 
lence classes of marked complete hyperbolic structures on int(M), not neces- 
sarily of finite area. The properness of the action of the mapping class group 
7To(Homeo(M)) on the Fricke space of M implies properness of the T -action 
of K - l (t). (For proof, see Section 2.2 of Abikoff [1], Bers-Gardiner [2] 6.5.6 
(page 156) of Buser [5], 6.3 of Imayoshi-Tanigawa [19], 2.4.1 of Harvey [16], or 
Section 2.7 of Nag [28].) 

When t = — 2, then K~ l (t) has five connected components. It is the union of 
the single (singular) point (0, 0, 0) and four copies of the Teichmiiller space 1m 
of M . Teichmiiller space Tm consists of equivalence classes of marked complete 
finite-area hyperbolic structures on M. The group T acts properly on n~ l {t) 
and fixes (0, 0, 0) . 

3.3 Singular hyperbolic structures on tori (— 2 < t < 2) 

We next consider the case 2 > t > — 2. Here n~ l (t) has five connected com- 
ponents. One component Ct is compact and consists of unitary characters, 
while the other four components correspond to singular hyperbolic structures 
on a torus with a cone point. Ct is diffeomorphic to S 2 , and the symplectic 
structure is a smooth area form. T acts ergodically on Ct- See Section 5 of 
Goldman [13] for a detailed discussion. 

Consider next the four noncompact components of n~ l (t). This case is similar 
to the previous case, except that the ends of M are replaced by cone points on 
a torus. Using the properness of the mapping class group action on Teichmiiller 
space, the action on these components of the relative character variety remains 
proper. However, none of the corresponding representations in Hom(7r, G) are 
discrete embeddings. Generically these representations are isomorphisms onto 
dense subgroups of SL(2, E) . 

We first show (Theorem 3.4.1) that every representation in these components 
is a lift (to SL(2,E)) of the holonomy representation of a singular hyperbolic 
structure. In Section 3.5 we deduce that V acts properly on the level sets K~ l {t) 
where -2 < t < 2. 
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3.4 Construction of hyperbolic structures on T 2 with one cone 
point 

Let 9 > and let denote the space of hyperbolic structures on T 2 with 
a conical singularity of cone angle 9. By results of McOwen [25] and Troy- 
anov [33] , identifies with the Teichmuller space X(#) , the deformation space 
of conformal structures on T 2 singular at one point with cone angle 9 . 

Theorem 3.4.1 Suppose £,r/ £ SL(2,R) and 7 := [£,,rj] is elliptic of rotation 
angle 9. Let M denote a one-holed torus whose fundamental group has the 
presentation (1.1.2). Then there exists a singular hyperbolic structure on M 
with a singularity of cone angle 9 having holonomy representation p defined by 

p(X) = p{Y) = r,, p(K) = 7. 
The proof will be based on the following. 

Lemma 3.4.2 Let p £ H 2 be a point fixed by 7 and consider the points 

Pa =P, 
Ps = V^Pi 
P2 = 

Pi = n^V^P- 

The four points Pi,P2,P3,P4 are the vertices of an embedded quadrilateral Q. 
In other words the four segments 

h = P1P2 
h = P2P3 
h =WP~a 
h = plpT 

are disjoint and bound a quadrilateral. 

Proof of Theorem 3.4.1 assuming Lemma 3.4.2 

£ maps p\ (respectively P2) to P4 (respectively p^). Therefore £ maps the 
directed edge l\ to I3 with the opposite orientation. Similarly n maps P2 
(respectively ^3) to p\ (respectively p^) so rj maps I2 to with the opposite 
orientation. 

From the embedding of Q and the identifications of its opposite sides by £ and 
rj, we construct a developing map for a (singular) hyperbolic structure on M as 

Qeometry & Topology, Volume 7 (2003) 



464 



William M Goldman 



follows. Let Q denote an abstract quadrilateral, that is, a cell complex with a 
single 2-cell, four 1-cells (the sides), and four 0-cells (the vertices). Denote the 
oriented edges Ci numbered in cyclic order and the vertices pi (i = 1,2,3,4) 
where 

dCi = {pi,p 2 }, d£ 2 = {P2,P3} 
9C 3 = {p3,Pa}, 8C 4 = {p4,Pi}- 
Let Ci denote Ci with the opposite orientation. Write 

Q = Q~ {Pl,P2,P3,P4}- 
Choose orientation-preserving homeomorphisms 

f : £1 — > C 3 , fj: C 2 — ► £4. 

Let II = (X, Y) denote the free group generated by X,Y . A model for the 
universal covering M of M is the quotient space of the Cartesian product 

Q x n 

by the equivalence relation generated by the identifications 

(x,rj) < — ► (£(x),Xr)) 

for x G C\ , and 

(x,r)) < > {rj{x),Yri) 

for x e C 2 ■ The action of II on Q x II defined by the trivial action on Q and 
right-multiplication on II descends to a free proper action of LT on M . This 
action corresponds to the action of deck transformations. Let p: II — > G be 
the homomorphism defined by: 

P(X) = £ 

P (Y) = n 

The embedding Q — ► Q together with the identifications of the sides of Q 
extends uniquely to a p-equivariant local homeomorphism M — ► H 2 . The 
holonomy around the puncture 

p([X,Y]) = [S,rj\=7 

is elliptic of rotation angle 9 fixing p. The resulting hyperbolic structure ex- 
tends to a singular hyperbolic structure on M with a singularity of cone angle 
9. Thus Theorem 3.4.1 reduces to Lemma 3.4.2. □ 

The proof of Lemma 3.4.2 is based on the following two lemmas. 
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Figure 5: A convex embedded quadrilateral 




Figure 6: When p 3 p^ meets pip 2 

Lemma 3.4.3 The points Pi,P2iPz-,Pa are not collinear. 

Lemma 3.4.4 If the sequence of points Pi,P2,P3,P4 are not the vertices of an 
embedded quadrilateral, then either l\ and I3 intersect, or I2 and I4 intersect. 

Proof of Lemma 3.4.3 Suppose that Pi,P2,P3,P4 are collinear. If more than 
one line contains Pi,P2,P3,P4, then all the points coincide and p fixes this point, 
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Figure 7: When P2P3 meets p^pi 




Figure 8: A nonconvex embedded quadrilateral 

contradicting 7 7^ I. Thus a unique line I contains pi,P2,P3,P4- We claim that 
I is p-invariant. 

If P3 = pa, then 77 fixes p and since 7 = fixes p, so must £r/£ _1 fix 

p. Since 7 = [£,77] is nontrivial, both £ and 77 are nontrivial. Thus £ maps the 
unique fixed point of 77 to the unique fixed point of £r?£ -1 , and both £ and 77 
fix p , contradicting 7 / I. Thus P3 7^ P4 and / is the unique line containing pj, 
and P4. 
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£(P2) = P3 and £(pi) = P4 imply that p\ 7^ P2- Thus I is the unique line 
containing pi and P2, and is £ -invariant. Similarly I is r/-invariant and thus 
p(tt) -invariant, as claimed. 

The commutator subgroup of the stabilizer of a line / consists of hyperbolic 
elements, contradicting ellipticity of 7. □ 

Proof of Lemma 3.4.4 If x, y G H 2 are distinct points, let xy denote the 
line passing through x and y. If z £ xy , let H x>y (z) denote the component (a 
half-space) of H 2 — xy containing z, and H XjV (z) denote the complementary 
component so that: 

H 2 = H XtV (z) Ux^U H x , y {z) 

Suppose first that no three of the points Pi,P2,P3,P4 are collinear. There 
are four cases, depending on where P4 lies in relation to the decompositions 
determined by the lines pip~2 and P2P3 '■ 

(1) p 4 e H P1P2 (p 3 ) n H P2P3 (pi) ; 

(2) p 4 G H P1P2 (p 3 ) n i7p 2P3 (pi ) ; 

(3) p 4 G H P1P2 (p 3 ) n (pi) ; 

(4) p 4 G i? PlP2 (p 3 ) n H P2P3 (pi ) . 

In the first case Pi,P2,P3,P4 are the vertices of a convex quadrilateral. (Com- 
pare Figure 5.) In the second case, P3P4 meets pTp~2- (Compare Figure 6.) 
In the third case, P2P3 meets pipT- (Compare Figure 7.) In the fourth case 
pi,P2,P3,P4 are the vertices of an embedded (nonconvex) quadrilateral. (Com- 
pare Figure 8.) 

Suppose next that three of the vertices are collinear. By Lemma 3.4.3, not all 
four vertices are collinear. By possibly conjugating p by £,77 etc., we may as- 
sume that pi,P2,P3 are collinear. If P2 lies between p\ and p 3 , then pi,P2,P3,P4 
are the vertices of an embedded quadrilateral (with a straight angle at P2 ). Oth- 
erwise P2P3 meets pipT at p\ . This completes the proof of Lemma 3.4.4. □ 

Returning to the proof of Lemma 3.4.2, we show that l\ and I3 cannot intersect; 
an identical proof implies I2 and I4 cannot intersect. 

Claim Suppose that l\ and I3 intersect. Then l\ n I3 is a point q and the 
triangles A(qpip4) and A(qp3P2) are congruent. 
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Proof of Claim If l\ n I3 is not a point, then it is a segment, contradicting 
Lemma 3.4.3. Since £(7i) = Z3 and r](l 2 ) = U, the lengths of opposite sides are 
equal: 

d(pi,P2) = Kh) = Kh) = d(p4,p 3 ), 
d(p2,ps) = t(h) = Z(h) = d(pi,p 4 ) 

Since the lengths of the corresponding sides are equal, the triangle A(p2PiP4) 
is congruent to A(p4p^p2). In particular 

AP2P1P4) = APiP3P2)- 

Similarly, Ap^PaPi) = ApiP2Pz)- Now 

AqPiPa) = AP2P\Pa) = APAP3P2) = AlP3P2) 

and similarly, AqPaPi) = AlPzPs)- Since U = pTPI is congruent to I2 = P3P2, 
triangles A(qp±p4) and A(qpsp2) are congruent as claimed. □ 

Now let I be the line through q bisecting the angle ApiQPs) such that reflection 
R in I interchanges A(qpip^) and A(qp?,p2) ■ Since 



R: < 



77: 



>P3 
>P4 
>Pl 

>P2, 

>p 4 
>P3 

Pi 

■PA, 



£ o R interchanges p% and p 4 , and Rot] interchanges P2 and p 3 . Since an 
orientation-reversing isometry of H 2 which interchanges two points must be 
reflection in a line, £, o R and R o rj have order two. Thus R conjugates £ to 
£ _1 and rj to rj~ 1 . 

One of two possibilities must occur: 

• At least one of £ and rj is elliptic or parabolic; 

• Both £ and rj are hyperbolic, and their invariant axes are each orthogonal 
to /. 

Neither possibility occurs, due to the following: 
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Lemma 3.4.5 Let £, 77 G SL(2, R) . The following conditions are equivalent: 

• tr[£,77] < 2; 

• £,77 are hyperbolic elements and their invariant axes cross. 

Proof Assuming tr[£,r?] < 2, we first show that £ and 77 must be hyperbolic. 
We first show that £ is not elliptic. If £ is elliptic (or ±1) , we may assume that 
£ £ SO (2) , that is, we represent £, rj by matrices 



008(6*) — sin(0) 
sm(8) cos(0) 





a 5 


> »7 = 


c <i 



where ad — be = I, whence 

tr[£,r?] =2 + sin 2 (6l)(a 2 + 6 2 + c 2 + (i 2 -2) > 2. 

Similarly if £ is parabolic, (2.3.1) implies that tr[£,?7] > 2. 

Thus £ is hyperbolic. Since [77, £] = [£,??] _1 , an identical argument shows that 
77 is hyperbolic. Denote their invariant axes by l^,l v respectively. 

It remains to show that tr[£, 77] < 2 if and only if n l v 7^ 0. 

By conjugation, we may assume that the fixed points of £ are ±1 and that 
the fixed points of 77 are r, 00. Thus n l v ^ if and only if — 1 < r < 1. 
Represent £, 77 by matrices 



cosh(6>) sinh(#) 
sinh(fl) cosh(0)_ 







-2r sinh(^) 



where 9,(j) j^O and 



tr[£,7?] = 2 + 4(r 2 - 1) sinh 2 (6») sinh 2 (0). 

Then tr[£,77] < 2 if and only if — 1 < r < 1 and tr[£, 77] = 2 if and only if 
r = ±1, as desired. □ 



This completes the proof of Lemma 3.4.2 (and also Theorem 3.4.1). □ 
3.5 Properness 

That T acts properly on K _1 (t) flE 3 now follows easily. The construction in 
Theorem 3.4.1 gives a map 

k- 1 ^) n [2,cx)) 3 — >%{6) (3.5.1) 
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which is evidently Out(7r)-equivariant. Every hyperbolic structure with conical 
singularities has an underlying singular conformal structure, where the singular- 
ities are again conical singularities, that is, they are defined by local coordinate 
charts to a model space, which in this case is a cone. However, there is an im- 
portant difference. Conical singularities in conformal structures are removable, 
while conical singularities in Riemannian metrics are not. 

Here is why conformal conical singularities are removable: Let D 2 C C be the 
unit disk and let 

S e ■ = {z G D 2 | < arg(z) < 9} 
be a sector of angle 9. Then 

Tig : Sg - {0} — ► D 2 - {0} 

z ^ z 2 ^ e 

is conformal. The model coordinate patch for a cone point of angle 9 is the cone 
Cg of angle 9, defined as the identification space Cg of Sg by the equivalence 
relation defined by 

z — e ±i9 z 

for z G dSg. That is, a cone point p has a coordinate patch neighborhood U 
and a coordinate chart tp : U — > Cg in the atlas defining the singular geometric 
structure. The power map Tig defines a conformal isomorphism between the 
Cg punctured at the cone point and a punctured disk (a "cone" of angle 2ir). 
Replacing the coordinate chart ip : U — > C at a cone point p of angle 9 by the 
composition Tig o ip gives a coordinate atlas for a conformal structure which is 
nonsingular at p and isomorphic to the original structure on the complement 
of p. 

The resulting map Z(9) — ► %m is evidently Out(7r)-equivariant. Since Out(7r) 
acts properly on Xm, Out(7r) acts properly on X(0), and hence on fl M 3 

as well. 

With more work one can show that (3.5.1) is an isomorphism. For any hyper- 
bolic structure on T 2 with a cone point p of angle < 9 < 2tt , the Arzela-Ascoli 
theorem (as in Buser [5] Section 1.5) applies to represent £,77 by geodesic loops 
based at p. Furthermore these geodesies intersect only at p. By developing 
this singular geometric structure one obtains a polygon Q as above. 

4 Reducible characters (t = 2) 

The level set k _1 (2) consists of characters of reducible representations. Over 
C such a representation is upper-triangular (2.3.2) with character defined by 
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(2.3.3). By (2.3.4), every (x,y,z) £ k 1 (2) lies in the image of the map 

$: C* x C* — > k _1 (2) (4.0.2) 



-i 



rj + rj 



The set of R -points k x (2) fit 3 is a singular algebraic hypersurface in 
with singular set 



So 



-2 
2 

-2 



Characters in So correspond to unipotent representations twisted by central 
characters (as in the sense of (2.2.1)). A central character is a homomorphism 
taking values in the center {±1} of G and a unipotent representation is a 
representation in ±U , where U is a unipotent subgroup of G. A reductive 
representation with character in Sq is itself a central character. The most 
general representation with character in Sq is one taking values in ±J7, where 
U is a unipotent subgroup of G. The character (2,2,2) is the character of 
any unipotent representation, for example the trivial representation. The other 
three points are images of (2,2,2) by the three nontrivial elements of S. 

The smooth stratum of k^ 1 (2) is the complement 

k- 1 {2) nR 3 -s . 

Denote its five components by Ck and Cj , where i = 0, 1, 2, 3. Here Cq denotes 
the component k^ 1 {2) n [2, oo) 3 and Cj = (crj)*Co where (<7j)* £ S is the sign- 
change automorphism fixing the i-ih coordinate (Section 2.2.1). 

The component Ck corresponds to reducible SU(2) -representations which are 
non-central, that is, their image does not lie in the center {±1} of SU(2). The 
closure of Ck is the union of Ck with Sq . The map 

U(l) x U(l) — C K 

v) i— + r 1 , + r?- 1 , er? + r 1 ^- 1 ) 

is a double branched covering space, with deck transformation 



and four branch points 



(Cr?) 



(±i,±i) 



which map to So . 
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2 4 

2 _ A\(„.2 



Figure 9: Level sets of (x — 4)(y 2 - 4) 

Similarly, each of the other components identifies with the quotient (lR + ) 2 /{±I} 
with the action of GL(2,Z). For example, Co is the image of the double 
branched covering 



+ X K + 



C 



(£, V)~(Z + C\V + n~\ HV + r V 1 ), 
with deck transformation 

For i = 1,2,3, the component Cj is the image of Cq under the sign-change 
(crj)*. Composing with the GL(2, Z) -equivariant diffeomorphisms 

R 2 — > R+ x Mi 



and 



(1.0) 1 — ► (exp(0,exp(?y)) 
2 — »U(1) x U(l) 



(I,??) 1 — ► (exp(27n£),exp(27TM7)) 
respectively, yields GL(2, Z) -equivariant double branched coverings 



d, and (R/Z 
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respectively. Furthermore these mappings pull back the invariant area form on 
k _1 (2) n R 3 to Lebesgue measure on R 2 . 

Since SL(2, Z) is a lattice in SL(2, R) and SL(2, R) acts transitively on R 2 with 
noncompact isotropy group, Moore's ergodicity theorem (Moore [27]; see also 
Feres [7], Zimmer [34] or Margulis [24]) implies that SL(2,Z) acts ergodically 
on R 2 . Thus GL(2,Z) acts ergodically on R 2 /{±I} and (R/Z) 2 /{±II} , and 
hence on each of the components 

c , d, c 2 , c 3 , c K ck- 1 (2) HR 3 . 

Since £ permutes Cq, C\, C 2 , C3 , the T-action on their union 

C U d U C 2 U C 3 = k' 1 {2) n R 3 - C K U So 
^ (M* x R*)/{±I} 

is ergodic. (In the case of Ck, any hyperbolic element in GL(2,Z) acts ergod- 
ically on U(l) x U(l) and hence on Co, a much stronger result.) 

5 Three- holed spheres and ergodicity (t > 2) 

Next we consider the level sets where t > 2. There is an important difference 
between the cases when t > 18 and 2 < t < 18. When t < 18, the T-action is 
ergodic, but when t > 18, wandering domains appear, arising from the Fricke 
spaces of a three- holed sphere P ("pair-of-pants"). The three- holed sphere 
is the only other orientable surface homotopy-equivalent to a one-holed torus, 
and homotopy equivalences to hyperbolic manifolds homeomorphic to P define 
points in these level sets when t > 18, which we call discrete P -characters 
However, the T -action on the complement of the discrete P-characters is er- 
godic. 

5.1 The Fricke space of a three-holed sphere. 

When t > 18, the octant 

n = (-00, -2) x (-00, -2) x (-00, -2) 

intersects K~ x (t) in a wandering domain and the images of ^0 n ^(t) are 
freely permuted by T . Let 

n = r-n - 

Characters in f2 correspond to discrete embeddings p: ir — ► SL(2,R) where 
the complete hyperbolic surface H 2 / p(ir) is diffeomorphic to a three-holed 
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sphere. We call such a discrete embedding a discrete P -embedding, and its 
character a discrete P -character. 

The fundamental group tti(P) is free of rank two. A pair of boundary compo- 
nents 81,82, an orientation on P , and a choice of arcs a±,ct2 from the basepoint 
to 81,82 determines a pair of free generators of tt\{P): 

X := (ai) -1 *<9i *ct\ 

Y := (o2) _1 * 82*02- 

A third generator Z := (XY)~ l corresponds to the third boundary component, 
obtaining a presentation of tti(P) as 

(X,Y,Z I XYZ = 1). 

Elements of are discrete P-characters such that the generators X, Y and 
Z := (IF) -1 of 7r correspond to the boundary components of the quotient 
hyperbolic surface H 2 / p(ir) . 

Lemma 5.1.1 A representation p € Hom(7r, SL(2, C)) has character [p] = 
{x, y,z) G T • f^o if and only if p is a discrete P-embedding such that X, Y, Z 
correspond to the components of 8H 2 /p(-jr). 

Proof The condition that [p] G Slo is equivalent to x,y, z < —2, which implies 
that the generators p{X) , p(Y) and p(XY) are hyperbolic and their invariant 
axes l p (x)Jp(Y)7^p(XY) are pairwise ultraparallel. Denoting the common per- 
pendicular to two ultraparallel lines /, by _L (/, /') , the six lines 

lp(X), -l-(lp(X),l p (Y)), lp(Y), ^-(lp(Y),lp(XY)), lp(XY), ^p{XY)^p(X)) 

bound a right-angled hexagon H . The union of H with its reflected image in 
_L (ip(x)Jp(Y)) is a fundamental domain for p(ir) acting on H 2 . (Goldman [11], 
Gilman-Maskit [10], 1-7, page 15). (Figure 10 depicts the identifications cor- 
responding to the generators p{X),p(Y).) The quotient is necessarily home- 
omorphic to a three-holed sphere P and the holonomy around components of 
8P are the three generators p{X), p(Y), p(XY) . □ 

Recall (Section 2.2.2) that V decomposes as the semidirect product 

r r (2) x 63. 

The mapping class group of a three- holed sphere P is isomorphic to Z/2X63. 
The 63 -factor corresponds to the group of permutations of ttq{8P). The 
Z/2 -factor is generated by the elliptic involution which acts by an orientation- 
reversing homeomorphism of P, whose fixed-point set is the union of three 
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Figure 10: Fundamental hexagons for a hyperbolic three-holed sphere 



disjoint arcs joining the boundary components. (See Section 2.2.3 for the cor- 
responding automorphism of it.) Accordingly, 63 preserves the Teichmiiller 
space of P. The following result shows that these are the only automorphisms 
of the character variety which preserve the discrete P-characters. 

Proposition 5.1.2 Q equals the disjoint union U 7 er (2) 7^0- 

Proof We show that if for some 7 € T( 2 ), the intersection Qq fl 7SI0 is 
nonempty, then 7 = 1. Suppose that [p] € fio H 7^0- By Lemma 5.1.1, 
both p and p o 7 are discrete P-embeddings such that l,y,(lF) _1 and 
7(X),7(T),7((Xy)- 1 ) correspond to dU 2 /p(7r). 

The automorphism 7 of the character space C 3 corresponds to an automor- 
phism 7 of 7T such that 

[p°i] = ^°t(M) 

for a sign-change automorphism cr* <G S . Thus p o 7 is also a discrete P- 
embedding, with quotient bounded by curves corresponding to j(X), ; y(Y) and 
j(XY). Then j(X) (respectively j(Y),j(XY)) is conjugate to X ±l (respec- 
tively Y^ 1 , (XY)^ 1 ). Such an automorphism is induced by a diffeomorphism 
of the three- holed sphere P . Since the elliptic involution generates the mapping 
class group of P, 7 must be an inner automorphism possibly composed with 
the elliptic involution of n, and hence must act trivially on characters. Thus 
7 = 1 as desired. □ 
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For t < 18, the domain fl does not meet (The closure Cl intersects 

k _1 (18) in the T-orbit of the character (—2,-2,-2) of the holonomy repre- 
sentation of a complete finite-area hyperbolic structure on P.) For t > 18, 
observe that K~ x (t) — ft contains the open subset 



Thus K~ 1 (t) — ft has positive measure in K _1 (t). 

Proposition 5.1.3 For any t > 2, the action of F on K _1 (i) — ft is ergodic. 

The proof uses an iterative procedure (Theorem 5.2.1) due to Kern-Isberner 
and Rosenberger [18], although their proof contains a gap near the end. See 
also Gilman and Maskit [10]. Theorem 5.2.1 is proved at the end of the paper. 

5.2 The equivalence relation defined by T 

Write u ~ v if there exists 7 G F such that ju = v. Since k is T-invariant, 
u ~ f implies that k(u) = k(u ) . 

Theorem 5.2.1 Suppose that u € E 3 satisfies k(u) > 2. Then there exists 
z) = u ^ u such that either 

• n G (—00, —2] 3 , in which case u is the character of a Fuchsian represen- 
tation whose quotient is a hyperbolic structure on a three-holed sphere 
P , with boundary mapping to either cusps or closed geodesies; 

• x £ [—2, 2] , in which case u is the character of a representation mapping 
X to a non-hyperbolic element. 

Recall that a measurable equivalence relation is ergodic if the only invariant 
measurable sets which are unions of equivalence classes are either null or conull. 
Equivalently an equivalence relation is ergodic if and only if every function con- 
stant on equivalence classes is constant almost everywhere. A group action de- 
fines an equivalence relation. However, equivalence relations are more flexible 
since every subset of a space with an equivalence relation inherits an equiva- 
lence relation (whether it is invariant or not). Suppose that S is a measurable 
subset of a measure space X . If every point in X is equivalent to a point of 
a measurable subset of S , then ergodicity of the equivalence relation on X is 
equivalent to ergodicity of S with respect to the measure class induced from 




X. 
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Proof of Proposition 5.1.3 assuming Theorem 5.2.1 For i = 1,2, let 
(E^ denote the subset of — where at least i of the coordinates lie 

in [—2,2]. Suppose that t > 2 and u E — Jl) . Then Theorem 5.2.1 

implies that T-u n (S^ 7^ 0. Thus ergodicity of the T-action on K _1 (i) — is 
equivalent to the ergodicity of the induced equivalence relation on (5^ . 

We now reduce to the equivalence relation on <B\ . By applying a permutation 
we may assume that — 2 < x < 2. 

The level set 

E(x ) := K~ l (t) nx _1 (x ) 

is defined by 

2 -^{y + Z ?+ 2 -^(y- Z ? = t-2 + xl 

and is an ellipse since — 2 < xq < 2. Furthermore the symplectic measure on 
n~ l {t) disintegrates under the map x: K _1 (t) — ► K to tx -invariant Lebesgue 
measure on E{x$) . In particular the Dehn twist tx (see Section 2.2.3) acts by 
the linear map 



y 




x —1 




y 


z 




1 




z 



which is linearly conjugate to a rotation of the circle through angle cos _1 (a;/2) . 
Thus for almost every xo £ (—2,2) the Dehn twist tx acts on E(xq) by a 
rotation of infinite order, and the action is ergodic. Furthermore by applying 
powers of tx , we may assume that — 2 < y < 2 as well. Thus ergodicity of the 
T -action on K~ l (t) — O is equivalent to the ergodicity of the induced equivalence 

(2) 

relation on <£\ ' . 

Since T acts by polynomial transformations over Z, those points of ft -1 (t) which 
are equivalent to a point with cos _1 (x/2) rational comprise a set of measure 
zero. We henceforth restrict to the complement of this set. 

The quadratic reflection (see Section 2.2.3) 





X 




X 


Q z : 


y 




y 




z 




xy — z 



is the deck transformation for the double covering of k 1 (t) given by projection 
R( x ,y) to the (x,y)-plane. The image 
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is the region 

Tit := { (x, y) € E 2 | (x 2 - 4)(y 2 - 4) + t - 2 > 0} 
and n^ ^) : k^ 1 ^) — ► TZt is the quotient map for the action of Q z . Thus 
ergodicity on (H^ reduces to ergodicity of the induced equivalence relation on 

IW(*; 2) ) = [-2,2] x [-2,2]. 

Ergodicity now follows as in Section 5.2 of [13]. Suppose that / : — — ► 

E is a T-invariant measurable function. The ergodic decomposition for the 
equivalence relation induced by the cyclic group (tx ) is the coordinate function 

x: [-2,2] x [-2,2] — [-2,2], 

and by ergodicity of (tx) on the level sets of x, there is a measurable function 
g: [—2,2] — > E such that / factors as / = gox almost everywhere. Applying 
the cyclic group (ry) to [—2,2], the function g is constant almost everywhere. 
Hence / is constant almost everywhere, proving ergodicity. □ 

5.3 The trace-reduction algorithm 

The proof of Theorem 5.2.1 is based on the following: 

Lemma 5.3.1 Let 2 < x < y < z. Suppose that K,(x,y,z) > 2. Let z' = 
xy — z. Then z — z' > 2^J k(x, y,z) — 2. 

The following expression for k(x, y, z) will be useful: 

n(x,y, z) - 2 = 1 {(2z - xy) 2 - (x 2 - 4)(y 2 - 4)} (5.3.1) 

= \{(z-z') 2 -(x 2 -4)(y 2 -4)}. 
For fixed x,y > 2, write K x , y : E — > E for the quadratic function 
Kx,y ■ z i — ► k(x, y, z) - 2 = z 2 - xyz + (x 2 + y 2 - 4). 

Then 

Kxl(i-™A)^J =[C-(x,y),C+(x,y)} 

where 

C±(x, y) = - (xy ± V(^ 2 -4)(y 2 -4)) . 
Furthermore reflection 

z i — ► z' = xy — z 
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interchanges the two intervals 



J_ = ( - oo, (-(x,y)) 

J+ = ((+(%, y), °o) 



comprising oo)) . 

Lemma 5.3.2 Suppose that 2 < x < y. Then £_(x,y) < y < ( + (x,y) 

Proof The conclusion is equivalent to n(x,y,y) < 2, which is what we prove. 
First observe that y > 2 implies y — l/2>3/2so that 



Conclusion of Proof of Lemma 5.3.1 

By Lemma 5.3.2, the quadratic function k XjV is negative at y. By hypothesis 
k X:V is positive at z >y. Therefore 

z > C+(x,y) > y. 

Thus z € J+ . Reflection z i — > z' interchanges the intervals J + and J_ , so 
z' G J_, that is z' < C-(^,y) < 2/ (Lemma 5.3.2). By (5.3.1), 

(z - z') 2 = 4 f y, z) - 2\ + (x 2 - 4) (y 2 - 4) 

> 4(/c(x, y,z) -2) 
whence (because z > y > z') 




and x < y implies that x + 2 — y 2 < y + 2 — y 2 <0. Therefore 

y, y) - 2 = x 2 + 2y 2 - xy 2 - 4 

= (x 2 - 4) + (2y 2 - xy 2 ) 
= (x-2)(x + 2-y 2 ) < 0. 



□ 



z — z' > 2y / '«(x, y,z) - 2, 



completing the proof of Lemma 5.3.1. 



□ 
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Conclusion of Proof of Theorem 5.2.1 Fix 

u = (x,y,z) G 1R 3 

with k(u) > 2. We seek u ~ u such that one of the following possibilities 
occurs: 

• one of the coordinates x, y, z lies in the interval [—2, 2] ; 

• x,y,z<-2. 

It therefore suffices to find u ~ u which lies in (— oo,2] 3 . Suppose that u does 
not satisfy this. The linear automorphisms in L are arbitrary permutations 
of the coordinates and sign-change automorphisms which allow changing the 
signs of two coordinates. By applying linear automorphisms, we can assume 
that 2 < x < y < z. 

Let (i := 2-^/ n(u) — 2 > 0. By Lemma 5.3.1 the quadratic reflection Q z G L 
given by 





X 




X 


u = 


y 




y 




z 




z' 



reduces z by more than fx. If z' < 2 then 

u ~ (-x,-y,z f ) G (-oo,2] 3 , 

completing the proof. Otherwise, all three coordinates of u are greater than 2 
so we repeat the process. Since each repetition decreases x + y + z by more 
than fj,, the procedure ends after at most (x + y + z — 6)//U steps. □ 
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Appendix: Elements of the modular group 

We describe in more detail the automorphisms V and interpret them geometrically in 
terms of mapping classes of M . 

Horowitz [17] determined the group Aut(C 3 ,«) of polynomial mappings C 3 — > C 3 
preserving k. We have already observed that the linear automorphisms form the 
scmidirect product £ xi 63 of the group £ = Z/2 x Z/2 of sign-changes (see Section 
2.2.1) and the symmetric group 63 consisting of permutations of the coordinates x,y,z. 
Horowitz proved that the automorphism group of (C 3 ,k) is generated by the linear 
automorphism group 

Aut(C 3 , k) n GL(3, C) = S xi 63 
and the quadratic reflection: 



X 




yz — x 


V 




V 


z 




z 



This group is commensurable with Out (71-). 

We denote this group by T ; it is isomorphic to a semidirect product 

T = PGL(2,Z) K (Z/2 ©Z/2) 

where PGL(2,Z) is the quotient of GL(2,Z) by the elliptic involution (see below) and 
(Z/2 © Z/2) is the group S of sign-changes. 
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A-l The elliptic involution However, Out(7r) does not act effectively on C 3 . To 
describe elements of Out(7r), we use the isomorphism h: Out(7r) — ► GL(2,Z) dis- 
cussed in Section 1.2. The kernel of the homomorphism Out(7r) — > Aut(C 3 ,«) is 
generated by I). 

The elliptic involution is a nontrivial mapping class which acts trivially on the character 
variety. This phenomenon is due to the hyperellipticity of the one-holed torus (as in 
[13], Section 10.2). The automorphism e of n given by: 

X i — > YX^Y- 1 ~ X- 1 
Y i — > (YX)Y- 1 (X- 1 Y- 1 ) ~ Y- 1 
XY i — > X^Y- 1 ~ (XY)' 1 

preserves K = [X, Y] and acts on the homology by the element 



-1 
-1 



G GL(2,Z) 



generating the center of GL(2,Z). Furthermore s acts identically on the characters 
(x, y, z) . Thus the homomorphism 

GL(2,Z) Out(Tr) — ► Aut(C 3 ,«) 
factors through PGL(2,Z) := GL(2,Z)/{±I}. 
Note, however, that e 2 = l k -i . The automorphism 

Lyxoe-.TT — >tt X\ — >X- X V — >Y^ XY< 
has order two in Aut(7r) but does not preserve K . 



X-'Y- 1 



(XY)- 1 



A-2 The symmetric group Next we describe the automorphisms of ir which cor- 
respond to permutations of the three trace coordinates x,y,z. Permuting the two 
generators X, Y gives: 



X i — > Y Y 
This automorphism P(\2) sends K 



X 

K- 1 



XY * — ► YX <~ XY. 
It acts on characters by: 





X 




y 


0P(i2))*: 


y 




X 




z 




z 



and on the homology by 
defined by the involution 



1 

1 



(13) 



Another transposition of the character space is 



: X i 
Y 
XY 



Y- l X- x 
XYX- 1 . 

x- 1 



Y 
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which maps K i — ► K 1 . It acts on characters by: 





X 




z 


0P(i3))*: 


y 




y 




z 




X 



and on the homology by 



-1 
-1 1 



. The involution 
: X < — > Y^XY 



(23) 



Y 
XY 



Y-'X- 1 
Y- 1 



maps K i — ► K 1 and acts on characters by: 





X 




X 


(-P(23))* : 


y 




z 




z 




y. 



and on the homology by 



1 -1 
-1 



. The composition f(i3) °-P(i2) w iH be denoted P(i23) 



since the composition of the transposition (12) with the transposition (13) in 63 equals 
the 3-cycle (123). Applying this composition, we obtain a 3-cycle of automorphisms 



X 
Y 
XY 



XYX- 1 
Y- X X- X 
XYX^Y^X- 1 



XYX- 1 Y- 1 {XY)- 1 

(XY)X(XY)- 1 

(XY)(X- 1 Y- 1 X)(XY)- 1 



which preserve K, although -P( 3 123 ) equals the elliptic involution e, not the identity. 
The action P(i2 3 ) on characters is: 



X 


^(123) 


z 




y 


y 




X 




z 


z 




y 




X 



which is the inverse of the permutation of coordinates given by (123) . The action on 
homology is given by the respective matrices: 



-1 

1 -1 



h(P(i 3 2)) 



-1 1 

-1 



In summary, we have the following correspondence between permutations of the coor- 
dinates and elements of PGL(2, Z) given by r 1— ► h(P T ) : 



(12) 



± 



1 

1 



(13) 



± 







(23) 



± 



-1 1 
1 
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(132) 



± 







(123) 



± 



1 

-1 1 



The isomorphism GL(2, Z/2) = 63 relates to the differential (at the origin) of the 
mappings in the image of Out(7r) — ► Aut(C 3 ,K) as follows. The origin is the only 
isolated point in the Out (n) -invariant set k _1 (— 2) n R 3 , and is thus fixed by all of 
Out(-7r). (The origin corresponds to the quaternion representation (2.6.1); see Section 
2.6.) Therefore taking the differential at the origin gives a representation 

Out(Tr) — ► GL(3,C) 
4> 1 — > rf(o,o,o)</ ) * 

whose image is 63 . In particular it identifies with the composition 



Out(Tr) GL(2,: 



GL(2,Z/2) = 63 



where the last arrow denotes reduction modulo 2. These facts can be checked by direct 
computation. 

A-3 A quadratic reflection Here is a mapping class corresponding to a reflection 
preserving (a;, y) . The automorphism Q z of it given by: 



maps K ■ 



X 
Y 1 
XY 1 

K- 1 and 



{XYX- 1 )X{XY~ 1 X- 1 ) - X 
{XYX- 1 )Y- 1 {XY~ 1 X- 1 ) - Y- 1 
(XYX- 1 )XY- 1 (XY- 1 X- 1 ) - XY' 1 





X 




X 


(Qz)*: 


V 




y 




z 




xy — z 



Similarly the other quadratic reflections are: 

Qy '■ 



{XY)X(Y- 1 X~ 1 ) - X 

(XY)X- 1 Y- 1 X- 1 (Y- 1 X- 1 ) -X^Y^X- 1 
Y^X- 1 = (XY)- 1 



which induces 



and 





X 




X 


(Qy)* '■ 


y 




xz — y 




z 




z 



KQv) = 




(XY)Y 2 X(Y- 1 X- 1 ) - Y 2 X 
(XYX- 1 )Y- 1 (XY- 1 X- 1 ) - Y- 1 
(XY 2 )(XY)(Y- 2 X- 1 ) - XY, 
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which induces 





X 




yz — x 


{Qx)* ■ 


V 




y 




z 




z 



,KQ X ) 



1 

2 -1 



The square of each of these three reflections is the identity element of Aut(7r). These 
reflections correspond to the generators (2.2.3) of the level-2 congruence subgroup 
GL(2,Z) (2) . 

A-4 Another involution The automorphism v 

X i — > Y- 1 

Y i — > YXY- 1 ~ X 
XY i — > XY- 1 
preserves K , satisfies v 1 = e and acts by: 





X 




y 


v. t : 


y 




X 




z 




xy — z 



1 

-1 



the composition of the transposition P{\2) and the quadratic reflection Q z . Note that 
(P(i2))* an d (Qz)* commute in Aut(C 3 ,«). 



A-5 A Dehn twist The automorphism r 

X i — > XY 



Y 
XY i 



Y 

XY 2 



preserves K , and acts by: 





X 




xy — z 


t* : 


y 




V 




z 




X 



h(r) = 



1 
1 1 



the composition fm) °Q Z - 
Similarly the Dehn twist Tx around X 

X 



Y 
XY 



preserves K, and acts by: 



the composition f (23) Qz ■ 



X 




X 


y 




xy — z 


z 




y 



X 

YX 

XYX 



h{T) = 



1 1 

1 
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